System for and method of intelligently directed segmentation analysis for automated microscope systems

ABSTRACT

The present disclosure includes systems and techniques relating to intelligently directed segmentation analysis for automated microscope systems. In general, in one implementation, the technique includes obtaining an image of at least a portion of a scan region including a biological specimen, partitioning the obtained image into zelles, determining one or more parameters of the zelles, performing a cluster analysis on the one or more parameters of the zelles, differentiating tissue of greater interest from tissue of lesser interest in the obtained image based on the cluster analysis and based on a test being performed for the biological specimen, and storing more information for the tissue of greater interest than information for the tissue of lesser interest. The cluster analysis can be a multivariate statistical cluster analysis, and the zelles can be test-dependent zelles (e.g., having dimensions defined according to the test being performed for the biological specimen).

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of the priority of U.S. ProvisionalApplication Ser. No. 60/705,096, filed Aug. 2, 2005 and entitled, Systemfor and Method of Intelligently Directed Segmentation Analysis forAutomated Microscope Systems.

TECHNICAL FIELD

The present application relates to acquiring and analyzing digitalimages viewed through a computer-controlled automated microscope, suchas using a computer-controlled automated microscope in an analysis ofbiological specimens.

BACKGROUND

The microscopic examination of tissue or tissue components is a commonand valuable practice in both medicine and biology. Such procedurestypically rely on the visual appearance of the tissue which is oftenenhanced by the use of specialized stains that bind to certain tissuecomponents, foreign bodies, or the products of cellular processes.

With the advent of computer technology, it has now become possible toautomate many of the manual examination procedures by using acomputer-controlled microscope to image a microscope slide, digitize theimages, and place them into the memory of a computer for subsequentanalysis, display, and storage.

Most automated microscope imaging systems do not have the ability topredict in advance what a pathologist or user may determine isinteresting or important on a slide containing a biological sample.Current automated systems, therefore, capture and store high-quality,images of the entire microscope slide. Using a high-power objective,such as a 60X or 100X objective, the field of view is quite smallrelative to the overall area of the biological sample. Therefore, thesystem takes many images to cover the entire area of the microscopeslide. Each high-power image requires significant storage capacity. Adatabase of information and images of an entire microscope slide canquickly begin to overwhelm available resources. In addition, only afraction of the information and images saved may be of any interest orbe used by a pathologist or other user in making a diagnosticevaluation. The remaining information and images may be of little to novalue.

Improving automated imaging systems depends on the ability of the systemto automate portions of the tissue examination process that wouldotherwise be manually performed by a pathologist or other user. This, inturn, relies on the ability of the system to identify not only thesample on the slide from other content, but to distinguish, withoutoperator intervention, those portions of the sample necessary for makinga diagnostic evaluation from those portions that are not needed. A highmagnification image can be stored for each confirmed object of interest.The images are then available for retrieval by a pathologist to reviewfor final diagnostic evaluation.

One current method of minimizing the storage capacity requirements of anautomated microscope imaging system is to use image compressiontechniques. An imaging system captures high-power images of the entiremicroscope slide and each image is then compressed in order to reduceits storage capacity requirements. However, most compression techniquesin use today utilize compression algorithms that are lossy and thus donot perfectly recreate the original image, i.e., the image suffers somedistortion during the compression algorithm's encoding and decodingprocess.

U.S. patent application Ser. No. 2003-0163031, entitled, Method andSystem for Providing Centralized Anatomic Pathology Services, describesa method of providing centralized anatomic pathology services. A masterstorage, such as a database, of pathology information is maintained, thestorage being accessible by pathologists associated with any regionallaboratory via a communications link. Tissue samples requiring pathologyprocessing are collected from a medical entity located in a firstgeographic region by a regional pathology laboratory. The tissue isprocessed, and a tissue slide is created. A digital, diagnostic qualityimage of the tissue slide is created and stored in the master storage. Apathologist, who may be remotely located with respect to the firstgeographic region, is provided with access to the stored diagnosticimage via the communications link, to enable diagnosis by thepathologist without physical possession of the slide. The digital,diagnostic quality image may be compressed before it is stored in themaster storage. The diagnosis may be a primary or a secondary diagnosis.In the case of a secondary or supplemental diagnosis, the pathologist isprovided with access to any prior analysis and annotations, stored inthe master storage, relating to the diagnostic image.

The '031 patent application, using wavelet compression, provides a meansto optimize image storage capacity at the master storage location andtransmission time across a communications link to a remote location.However, the wavelet compression method of the '031 patent applicationutilizes a lossy compression algorithm, whereby thecompressed/decompressed image is different from the original digitalimage. These differences, though they may be subtle, could inhibit apathologist from making an accurate diagnostic evaluation.

U.S. Pat. No. 6,272,235, entitled, Method and Apparatus for Creating aVirtual Microscope Slide, describes a method and apparatus forconstructing a virtual microscope slide made up of digitally scannedimages from a microscope specimen. The digitally scanned images arearranged in a tiled format convenient for viewing without a microscope,and for transferring the tiled images for viewing by another at a remotelocation. Several original microscope views at a low magnification aredigitized and stored as digitized images coherently seamed togetherto-provide an overall virtual, macro image of the specimen at a lowerresolution. Several original microscope views at higher magnificationsare digitized and stored as digitized images coherently seamed togetherto provide virtual micro images at higher resolution. A data structureis formed with these virtual macro and micro digitized images along withtheir mapping coordinates. Preferably, a generic viewing program is alsoprovided in the data structure that allows remote users to manipulateand interpret the tiled images on the user's monitor. Also, the datastructure is formed with significantly compressed data so as to betransmitted over low bandwidth channels, such as the Internet, withoutloss of resolution that would interfere with the analysis at aremotely-located pathologist receiving the data structure over theInternet. The preferred interactive program allows the pathologist toscroll and view neighboring image areas of interest. A marker on themacro image indicates to the user the location of the micro image andassists the user in selecting areas from the macro image to be viewed athigher resolution and magnification.

While the '235 patent provides a means to selectively capture digitalimages at high magnification, rather than image the entire microscopeslide using a high power objective, it relies on the user of theapparatus to manually select a region of interest for high magnificationdigital imaging. A user examines the macro image or original specimenfor significant details. Typically, the user will highlight with amarking pen the areas to be viewed at higher magnification. The userthen changes the magnification to a higher power objective, moves themicroscope slide to bring the selected region into view, and begins tocapture images of the selected region.

U.S. Pat. No. 5,978,498, entitled, Apparatus for AutomatedIdentification of Cell Groupings on a Biological Specimen, describes thedetection of cellular aggregates within cytologic samples. An imageanalysis system with an image gathering system includes a camera, amotion controller, an illumination system and an interface; the systemobtains images of cell groupings. The image gathering system isconstructed for gathering image data of a specimen mounted on a slideand is coupled to a data processing system. Image data is transferredfrom the image gathering system to the data processing system. The dataprocessing system obtains objects of interest. A four step process findscellular aggregates. The first step is acquisition of an image foranalysis. The second step is extraction of image features. The thirdstep is classification of the image to determine if any potentialcellular aggregates may exist in the image. The fourth step issegmentation of objects which includes the substeps of detecting andlocating potential cellular aggregates.

SUMMARY

The present disclosure includes systems and techniques relating tointelligently directed segmentation analysis for automated microscopesystems. Implementations of the systems and techniques described heremay occur in hardware, firmware, software or combinations thereof, andmay include computer program instructions for causing a programmablemachine to perform the operations described.

According to some implementations, an automated imaging system includesa microscope, a controller coupled with the microscope, and a displaydevice coupled with the controller. The microscope can be acomputer-controlled microscope electrically connected to the controllerand including a barcode reader, a camera, a serial interface, one ormore sensors, on or more motors, a light source, a turret, and a datainterface. The controller can be configured to operate the microscopeautonomously, to present image data on the display device, and toperform a cluster analysis operation.

The method of performing the cluster analysis operation can includesetting operating parameters, performing a silhouette scan, calculatingzelle parameters, performing cluster analysis on a set of zelleparameters, segmenting the zelles into groups, determining whichresulting groups are valuable and which are not, determining how manyhigh-power images to capture of each valuable cluster, capturinghigh-power images, and storing the images and associated data. Themethod can further include codifying knowledge derived from userinteraction with the system, where this codifying can include displayinga low-magnification image, selecting a region to view athigh-magnification, determining whether an image has been captured andstored of the selected location, displaying a high-magnification imageof the selected location if it exists, or indicating the location of themost similar image that has been captured and stored, displaying thehigh-magnification image if the alternative location is acceptable,capturing and displaying a high-magnification image of the originallyselected location if the alternative location is not acceptable, andstoring attributes and parameters of the high-magnification image tobuild a knowledge-base that optimizes future use of the automatedmicroscope imaging system.

According to further implementations, various methods of analyzing imagedata can be effected in a system, apparatus or article including amachine-readable medium storing instructions operable to cause one ormore machines to perform operations of the method. For example, anapparatus can include an interface configured to connect with amicroscope, and a controller configured to send signals through theinterface to operate the microscope and to perform the operations of amethod. The interface can include a serial interface and a datainterface, and the controller can be a special-purpose or conventionalcomputer.

The method can include obtaining an image of at least a portion of ascan region including a biological specimen, differentiating tissue ofgreater interest from tissue of lesser interest in the image based on atest being performed for the biological specimen and based on a clusteranalysis of data from the image, and storing information for the tissueof greater interest, which falls in the scan region. The differentiatingcan include defining subimages in the image based on the test beingperformed for the biological specimen, determining one or moreparameters of the subimages, performing the cluster analysis on the oneor more parameters of the subimages, and identifying one or more areasin the subimages based on results of the cluster analysis, the one ormore areas including the tissue of greater interest.

Obtaining the image can include performing a silhouette scan. Theinformation storing can include retaining high resolution data for thetissue of greater interest, and discarding high resolution data for thetissue of lesser interest.

The defining subimages can include specifying subimage dimensions basedon the test being performed for the biological specimen. The determiningcan include determining multiple parameters of the subimages, and theperforming can include performing a multivariate statistical clusteranalysis on the multiple parameters. The identifying can includeselecting a proper subset of resulting clusters based on operatingparameters set for the test being performed for the biological specimen.The method can further include obtaining one or more highermagnification images of the biological specimen in the one or more areasof the subimages, and the information storing can include saving the oneor more higher magnification images.

The method can include obtaining one or more higher magnification imagesamples of a cluster until a predefined number of samples meeting aspecified criteria have been obtained for the cluster. The method caninclude obtaining a higher magnification image sample of a cluster, theinformation storing can include saving the higher magnification imagesample along with a lower magnification image of the cluster andinformation linking the higher magnification image sample with the lowermagnification image, such that the higher magnification image sample isreturned in response to a request for a high power image of a region inthe lower magnification image, wherein the region does not overlap withthe higher magnification image sample but is statistically similar tothe higher magnification image sample according to the cluster.Obtaining the higher magnification image sample can include obtainingmultiple samples covering representative members of the cluster, and theinformation storing can include saving the samples, the lowermagnification image and the linking information in a single file fordistribution.

The method (which can be realized in a system, apparatus or article) caninclude obtaining an image of at least a portion of a scan regionincluding a biological specimen, subdividing the obtained image into aplurality of subimages, wherein the subdividing is based on a test beingperformed for the biological specimen, generating a derivative imagewherein image units of the derivative image are derived from respectiveones the subimages, performing an automated analysis of the derivativeimage to identify one or more areas of interest for the test, andstoring information for the one or more areas of interest, which fall inthe at least a portion of the scan region. Performing the automatedanalysis can include performing a multivariate statistical clusteranalysis, and grouping quadrants of the obtained image based on resultsof the multivariate statistical cluster analysis and the test beingperformed for the biological specimen. Subdividing the obtained imagecan include specifying subimage dimensions based on the test beingperformed for the biological specimen.

The grouping can form groups of quadrants, and the performing theautomated analysis can include selecting a proper subset of the groupsbased on the test being performed for the biological specimen toidentify the one or more areas of interest. The grouping can form groupsof quadrants, the performing the automated analysis can includedetermining a number of sample locations covering representative membersof the groups based on the test, and the information storing can includesaving lower magnification image data for the groups and highermagnification image data for the sample locations.

The method can include returning, in response to a request for a highpower image of a first region in the lower magnification image data, atleast a portion of the higher magnification image data corresponding toa second region with similar characteristics to the first regionaccording to the multivariate statistical cluster analysis. The canfurther include updating a knowledge-base according to user inputprovided with respect to the at least a portion of the highermagnification image data returned, wherein the updated knowledge-baseaffects future applications of the multivariate statistical clusteranalysis for the test.

The method (which can be realized in a system, apparatus or article) caninclude obtaining an image of at least a portion of a scan regionincluding a biological specimen, partitioning the obtained image intozelles, determining one or more parameters of the zelles, performing acluster analysis on the one or more parameters of the zelles,differentiating tissue of greater interest from tissue of lesserinterest in the obtained image based on the cluster analysis and basedon a test being performed for the biological specimen, and storing moreinformation for the tissue of greater interest than information for thetissue of lesser interest. The cluster analysis can include amultivariate statistical cluster analysis, and the zelles can includetest-dependent zelles. The method can include segmenting the zelles intoclusters exhibiting similar characteristics among a portion of the oneor more parameters determined to best cluster the zelles according tothe cluster analysis. The method can further include determining whichclusters contain zelles that most likely contain content that isvaluable to a pathologist in making a diagnostic evaluation.

The method can include determining how many high-power images of zellesto retain for each cluster based on a knowledge-base codifying previoustest experience. The method can include capturing the high-power imagesof zelles, analyzing the captured high-power images to determine if theymeet specified criteria, and terminating the capturing once a sufficientnumber of high-power images have been acquired for the test according tothe determining how many high-power images of zelles to retain.

The method can include presenting on a display device, in response to arequest for a high power image of a first region, at least a portion ofone or more of the high-power images, the at least a portioncorresponding to a second region with similar characteristics to thefirst region according to the cluster analysis. Furthermore, the methodcan include updating the knowledge-base according to user input, whereinthe updated knowledge-base affects future applications of the clusteranalysis for the test.

One or more of the following advantages may be provided. A pathologistcan be provided for viewing, in order to make an accurate assessment ofa tissue sample or other specimen, one or more appropriate high-power,undistorted images. Images need not be compressed using lossycompression algorithms as only the most relevant portions of a slide canbe scanned at high resolution as needed. Rapid data collection andreduced need for processing power in the automated microscope system canbe realized. Digital images can be rapidly captured and stored, whileminimizing storage requirements, yet preserving the quality of theoriginal image.

The stored images on an automated microscope system allow a pathologistor other user to automate the process of finding, focusing, and viewinga specimen on a microscope slide. The typical process involves lookingat the slide at a low magnification to determine where on the slideinteresting samples are located, moving the slide such that aninteresting area is centered in the field of view, increasing the powerof the objective, and viewing the sample at a higher power to confirm itis of genuine interest and to make a diagnosis. An automated microscopesystem according to the present disclosure may determine where aspecimen of interest lies on a microscope slide and also determine whatportions of that specimen a pathologist or other user may findinteresting. This can result in significant savings in processing timeand image storage requirements as high-power images of every location ona microscope slide need not be captured and saved.

In many cases, the area of interest on a slide represents only afraction of the entire sample. The present systems and techniques can beused to predict what areas of a microscope slide a pathologist may needto view, and to capture and store digital images of only thoselocations. An automated system can quickly identify and capture digitalimages of only those areas a pathologist or other user may need to viewunder higher magnification in order to complete a diagnosis, which canresult in significant processing and storage efficiencies. Inparticular, the present systems and techniques can be used to minimizethe storage requirements of digital microscopic images in such a waythat the quality of the images is maintained, to minimize theacquisition time of digital microscopic images at high magnificationobjectives, and to identify areas of interest on a microscope slide tocapture and store images at a high magnification objective.

Details of one or more embodiments are set forth in the accompanyingdrawings and the description below. Other features and advantages may beapparent from the description and drawings, and from the claims.

DRAWING DESCRIPTIONS

These and other aspects will now be described in detail with referenceto the following drawings.

FIG. 1 is a block diagram showing a microscope imaging system, accordingto some implementations.

FIGS. 2A and 2B show an expanded view of interesting and non-interestingregions of a microscope slide including area partitioning, according tosome implementations.

FIG. 3 is a flow diagram of a method of performing a cluster analysisoperation that can serve to optimize image data acquisition time anddata storage, according to some implementations.

FIG. 4 shows segmentation of zelles across two variables andsegmentation of data into discrete groups or clusters, according to someimplementations.

FIG. 5 is a flow diagram of a method of codifying knowledge that canserve to optimize image data acquisition time and data storage,according to some implementations.

DETAILED DESCRIPTION

The systems and techniques described here relate to capturing andstoring digital images for use with a computer-controlled automatedmicroscope imaging system. A method can include analyzing alow-magnification image of the entire microscope slide to identify areasof interest. Next, quadrants that exhibit similar characteristics can begrouped together. Images of a subset of quadrants from each group canthen be captured and stored at high magnification. In this manner, thedata acquisition time and the image data storage requirements can beoptimized, while still satisfying the needs of the pathologist or otheruser.

FIG. 1 illustrates a high-level functional diagram of a microscopeimaging system 100. Microscope imaging system 100 is representative of ageneralized imaging system suitable for use with the optimized imageacquisition techniques described in detail in connection with FIGS. 2-5.Microscope imaging system 100 includes a microscope 110 that iselectrically connected to a controller 112 that has a display device114. Controller 112 is representative of any special-purpose orconventional computer, such as a desktop, laptop, or host computer.Controller 112 can be loaded with the appropriate software forcontrolling microscope imaging system 100, such as software for runningimage-processing algorithms and image analysis algorithms. Displaydevice 114 can be any special-purpose or conventional display device(e.g., a computer monitor) that outputs graphical images to a user.

Microscope 110 is a computer-controlled microscope suitable for use inan automated imaging system. An example of microscope 110 is aChromaVision Automated Cellular Imaging System (ACIS). Microscope 110can further include a barcode reader 116, a camera 118, a serialinterface 120, one or more sensors 122, one or more motors 124, a lightsource 126, a turret 128, and a data interface 130.

Barcode reader 116 is a standard barcode reader capable of detecting anidentifier upon, in the example of microscope imaging system 100, astandard microscope slide (not shown). Camera 118 is a digital camerathat has selectable resolution capabilities. Camera 118 is mounted uponturret 128 of microscope 110, such that its aperture is aligned with thefield of view (FOV) of any lens associated with turret 128. Barcodereader 116 and camera 118 can feed electrical inputs of serial interface120, which facilitates a serial communication link between theseelements and controller 112. For example, serial interface 120 canprovide a USB (Universal Serial Bus) connection to controller 112.Furthermore, camera 118 can provide a direct video output connect to avideo card (not shown) within controller 112 that gathers the image datafrom camera 118 for processing.

Sensors 122 include, but are not limited to, position sensors,temperature sensors, and light intensity sensors or optical encoders.Motors 124 can be conventional servomotors associated with the motioncontrol of microscope 110, such as for rotating the appropriatelypowered lens within the optical path of microscope 110, for adjustingfocus, or for controlling an automated microscope stage (not shown).Light source 126 can be any suitable light source for appropriatelyilluminating the FOV of microscope 110, such that the creation of adigital image of that FOV is possible. Turret 128 can be a conventionalmotor-driven microscope turret, upon which is mounted a set of lenses ofvarying power that may be rotated into the optical path of microscope110. Turret 128 is also suitably controlled to provide the desiredfocus. Sensors 122, motors 124, light source 126, and turret 128 canfeed electrical inputs of data interface 130. Data interface 130 can bea conventional system driver card, which facilitates a datacommunication link between these elements and a motion control card (notshown) within controller 112.

Although specific functions of microscope imaging system 100 are furtherdescribed in reference to FIGS. 2 through 5, the generalized operationof microscope imaging system 100 is described in reference to FIG. 1, asfollows. A continuous supply of standard microscope slides that have abiological sample deposited thereon is fed to the automated microscopestage of microscope 110 via an in-feed stage and, subsequently, ispositioned in the FOV of microscope 110. Additionally, during thetransition from the in-feed stage of microscope imaging system 100 tothe microscope stage of microscope 110, the identifier (ID) of thetarget microscope slide is read by barcode reader 116. The target slideis subsequently scanned at various resolutions and magnifications, basedon image-processing algorithms and image analysis algorithms executed bycontroller 112. Upon completion of the image scan operation, the slideis transferred out of microscope imaging system 100 via an out-feedstage (not shown), the slide ID and image data for that particular slideis transmitted to controller 112 and stored in memory, and the motioncontrol system moves the next target slide into the FOV of microscope110.

This process automatically repeats for each microscope slide that isautomatically fed into microscope imaging system 100. It is noted thatmicroscope imaging system 100 operates autonomously, i.e., a cliniciancan initiate microscope imaging system 100 and microscope imaging system100 can subsequently operate automatically without human intervention,so long as a supply of microscope slides is available at its in-feedstage and no system errors occur. At any time, however, a clinician mayview and/or manipulate the digital image of any given slide viacontroller 112 and display device 114 for the inspection and analysis ofany given specimen, as is well known in anatomic pathology. This ispossible because controller 112 can reconstruct the image by using theimage data associated with the contiguous FOVs and the imageregistration information.

FIG. 2A illustrates an expanded view of a microscope slide 200 andspecimen resulting from microscope imaging system 100 having performed asilhouette scan operation, as detailed in U.S. patent application Ser.No. 10/413,493 (U.S. Pub. No. 2004-0202357 A1), filed Apr. 11, 2003, andentitled, Silhouette Image Acquisition, which is hereby incorporated byreference. FIG. 2A illustrates that microscope slide 200 can bepartitioned into an array of contiguous segments or zelles 210 coveringthe entire area of microscope slide 200. The area of each zelle 210 canbe defined by the power (i.e., magnification) setting of microscope 110.Those skilled in the art will appreciate that a microscopic FOV reducesvery substantially as the magnification increases. Zelles 210 mayoverlap slightly or abut. Overlap may be useful to ensure that no regionis missed due to mechanical inaccuracies in the X, Y stage of microscope110, and depending upon the smallness of the expected target. Zelles 210can be representative of the FOVs, in which low magnification andresolution are used; thus, operation time and the amount of stored imagedata can be minimized. Additionally, a low-power lens has a greaterdepth of focus, so that microscope 110 can search for tissue withouthaving to refocus. The low-power lens of microscope 110 can be focusedat either a best-guess z-plane or a z-plane derived from microscopecalibration. Moreover, the present systems and techniques can employ thefocusing systems and techniques described in U.S. Patent Application No.TO BE DETERMINED, filed MONTH DAY, YEAR, and entitled, System for Methodof Focusing for Automated Microscope Systems, which is herebyincorporated by reference.

Any zelle 210 found to have specimen content can be classified asinteresting and mapped as a logical 1. By contrast, any zelle 210 foundto have no specimen content can be classified as non-interesting andmapped as a logical 0. In this manner, a silhouette of the specimen,i.e., a sample 230, is formed, as shown in FIG. 2A, thereby creatingwhat is effectively a low-resolution image that may be processed usingstandard image-processing algorithms. An image table can be generatedthat represents the low-resolution image of sample 230.

Parameters are set depending on the test and application for analyzingeach zelle 210 and determining whether there is anything of interest ineach zelle 210. A statistical algorithm (e.g., a multivariatestatistical cluster analysis) can be run to determine whether there isanything of interest in each zelle 210. The classification of areas ofinterest can be uniquely peculiar to each particular application (test).For example, a priority may be set for blue stain, red stain, any speckof tissue, or specifically a large volume of tissue to be classified asinteresting. Consequently, the biological requirements of eachparticular test determine what is of interest and, thus, determine theparameters. Therefore, each zelle 210 can be analyzed usingpredetermined parameters for a particular test using associatedalgorithms that determine whether what is found in each zelle 210matches the predetermined criteria and is therefore classified asinteresting.

Microscope slide 200 can be further partitioned. Statistically, basedupon the entire area of a particular zelle 210, there may be very littlematerial of interest within zelle 210. One way to handle this is toarbitrarily subdivide each zelle 210 into yet smaller regions using dataprocessing. As an example, FIG. 2A illustrates that each zelle 210 canbe further partitioned into an array of contiguous minor-zelles 220,thereby forming an array of minor-zelles 220 covering the entire area ofmicroscope slide 200.

Like zelles 210, the interestingness of these yet smaller minor-zelles220 can be assessed via image-processing algorithms and image analysisalgorithms. This can be done in preparation for a future operation ofcollecting and saving a series of higher magnification images, as onlythe interesting regions may need be examined at a higher magnification.Although not a requirement, minor-zelles 220 may be designed to matchthe anticipated FOVs of the higher power images of later phases ofoperation. The size of minor-zelles 220 and whether minor-zelles 220match up with the anticipated FOVs of the higher power images can beadjustable system parameters.

Using image-processing algorithms and image analysis algorithms (whichcan be executed by controller 112) minor-zelles 220 found to havespecimen content can be classified as interesting and mapped as alogical 1. By contrast, any minor-zelles 220 found to have no specimencontent can be classified as non-interesting and mapped as a logical 0.In this manner, a yet more precise silhouette of sample 230 can beformed, thereby creating a slightly higher-resolution image that may beprocessed using standard image-processing algorithms. It is furtherunderstood that silhouette scan could continue to process microscopeslide 200 further into yet smaller minor zelles.

In general, any acquired image (whether it be a single FOV image or acomposite image stitched together from multiple FOVs) can be subdividedinto a plurality of subimages based on the test being currentlyperformed for the biological specimen 230. The dimensions of thesubimages can be specified by the parameters of the test. For example,the test can specify a size for the zelles used in the cluster analysisdescribed below, and these test-dependent zelles 240 can be arranged tofully cover the specimen 230 based on the minor zelles 220 found in thesilhouette scan (as shown in FIGS. 2A and 2B). Thus, the size of thezelles and whether the zelles match up with the anticipated FOVs forlater image acquisition are adjustable system parameters, which candepend on the test being run. Likewise, the test-dependent zelles can beused to derive a new image 250 that may be processed using variousimage-processing algorithms; standard image-processing algorithms, suchas those that dilate, erode, or assess whether pixels of an image areisolated or connected, can be run on the derivative image 250. Thus, thezelles can be understood as source data for pixels of a new image to beprocessed, and the pixels of this new image indicate certaincharacteristics of the tissue in the corresponding zelles as determinedby the derivation procedure.

After image tables are generated, the image data associated withnon-interesting regions may be discarded, thereby optimizing the storagespace requirements upon controller 112. Furthermore, after image tablesare generated, microscope imaging system 100 may perform anyimage-processing and analysis algorithm as desired. For example,microscope imaging system 100 can determine whether microscope slide 200contains a valid sample 230 or whether sample 230 meets distributionexpectations, and so on.

FIG. 3 illustrates a flow diagram of a method 300 for performingintelligent, directed segmentation that optimizes image data acquisitiontime and image data storage requirements. By first forming groups ofzelles with similar characteristics and then imaging (or retaining) onlya few representative zelles of each group, the data acquisition time andimage data storage requirements can be significantly improved.

At 305, operating parameters can be set. Microscope imaging system 100can set operating parameters that are used to enhance the operation ofsubsequent operations and improve the confidence of the resultingclusters or groupings. Operating parameters can include functions,constants or inputs, which can be derived from rules-of-thumb that arebased upon prior knowledge and attributes of the microscope slide 220,cover slip 240, or sample 230. For example, parameters may include theexpected specimen color, size, features, or distribution, and may alsobe related to the test being run. Parameters may also include thethickness of microscope slide 200 or of sample 230. operating parameterscan be derived by associating prior knowledge data already stored withinmicroscope imaging system 100 with a unique identifier on microscopeslide 200, such as a barcode. In addition, a user may manually enteradditional information or parameters directly via a graphical userinterface designed for accepting operating parameters to microscopeimaging system 100. Operating parameters may also be derived from aknowledge base that is continuously updated. As the knowledge basegrows, the system learns from previous results and improves theoperation of subsequent tests. The system can thus become more efficientand the results can be more accurate over time. The process of updatingthe knowledge base is described in further detail below in connectionwith FIG. 5.

At 310, a silhouette scan can be performed. Microscope imaging system100 can differentiate interesting vs. non-interesting tissue (e.g., fattissue versus stroma based on color and texture, or tumor versus notumor based on color and texture) in areas (e.g., zelles 210, 220 or240) in a high-resolution, low-magnification image. Microscope slide 200(or just the portions of the slide 200 that contain tissue) can bepartitioned into an array of contiguous segments, or zelles, that coverthe entire area. The area of each zelle can be defined by the power(i.e., magnification) setting of microscope 110, by the test beingperformed, or a combination of these. The X, Y coordinates of each zellecan be captured and stored in the memory of microscope imaging system100. The interesting zelles are those that have a higher probability ofcontaining the biological specimen. These zelles have one or morestatistics or parameters that have exceeded a specified threshold.

At 315, zelle parameters can be calculated. Microscope imaging system100 can collect and calculate data for each interesting zelle. Stored inthe memory of microscope imaging system 100, the data can be associatedwith a particular zelle from which it came. Data or parameters collectedon an interesting zelle may include location coordinates, lightness anddarkness, color, variation in the color, edginess, variation/standarddeviation of power of the pixel values (brightness), different colorsrepresented and ratios of colors, number of non-white pixels, number ofnuclei, or average size of nuclei.

At 320, a cluster analysis can be performed. Microscope imaging system100 can analyze the parameters calculated at 315 from each zelle todetermine how well a particular parameter-is able to cluster the zelles.A cluster is a group of zelles that exhibit similar characteristics in aparticular parameter or set of parameters. For example, FIG. 4 depicts atwo-dimensional plot of segmenting data 400, which is representative ofsample data analyzed across the two-dimensional brightness-color space.For the purposes of simplicity, the data is depicted only in thetwo-dimensional space. However, in practice, zelles may be analyzed in a3+ multi-dimensional space, wherein each dimension corresponds to one ofthe parameters calculated at 315. In FIG. 4, each zelle has a scorecorresponding to its average brightness and a score corresponding to itsaverage color. If one were to analyze the zelles by only the brightnessparameter, it would be readily apparent that the zelles are uniformlydistributed across a broad range. No discernable groups would beevident. If the zelles were analyzed by only the color parameter, twogroups could be recognized. One group would contain the zelles of aCluster I in FIG. 4 and another group would contain the zelles ofClusters II and III in FIG. 4. By analyzing the data across both thebrightness and color dimensions together, three discrete clusters areevident, as indicate in FIG. 4. A cluster analysis algorithm can takeinto account a parameter's ability to cluster the data on its own aswell as with other variables. The results of the cluster analysis allowmicroscope imaging system 100 to determine which parameters are able tocluster the data and which parameters provide no helpful clusterinformation. Operating parameters set at 305 can provide to microscopeimaging system 100, in advance, the zelle parameters expected to bestcluster zelles for a particular specimen type or diagnostic test,further optimizing the efficiency of microscope imaging system 100.Those parameters that are not able to provide any meaningful informationcan be ignored, which can further improving the efficiency and speed ofthe processing.

At 325, the data can be segmented. Microscope imaging system 100 canform groups or clusters of zelles exhibiting similar characteristicsamong those parameters determined at 320 to best cluster the data. Forexample, FIG. 4 depicts sample data segmented by brightness and color.Three clusters are readily identifiable. Cluster II includes thosezelles that are very bright but contain very little color; Cluster Iincludes those zelles that are not quite as bright but have quite a bitmore color; and Cluster III includes those zelles that are neitherbright nor contain much color. This example, for the purposes ofsimplicity, shows zelles segmented in the two-dimensionalbrightness-color space. However, in practice, zelles may be segmented inthe n-dimensional space, where n is the number of parameters determinedat 320 to be useful in creating clusters.

At 330, a determination can be made as to which clusters are valuableand which are not. Microscope imaging system 100 can use operatingparameters set at 305 to determine which clusters contain zelles thatmost likely contain content that is valuable to a pathologist in makinga diagnostic evaluation. There may be several clusters formed as aresult of the data segmentation at 325. One or more of these clustersmay not contain any valuable information a pathologist needs inanalyzing sample 230. However, one or more other clusters may containzelles critical to analyzing sample 230 and making an accuratediagnostic evaluation. Working with only those clusters determined tocontain valuable content allows data acquisition time and image datastorage requirements to be optimized, while still satisfying the needsof the pathologist or other user. The knowledge-base, further explainedin method 500, can assist microscope imaging system 100 in determining,in advance, which clusters most likely contain valuable content. It isfurther understood that if little or no a priori knowledge is availableas operating parameters, microscope imaging system 100 can determinethat all clusters are valuable by default.

At 335, a determination can be made as to how many high-power images ofeach cluster to capture. Using operating parameters at 305, microscopeimaging system 100 can determine how many high-power images of zelles tocapture from each valuable cluster as determined at 330. By capturingand storing images of only a sample of zelles that fall into one of theidentified clusters, method 300 can reduce the image storage capacityrequirements, increase the speed and efficiency of the process, andensure high-power images are available for the interesting areas apathologist is likely to want to view. In some embodiments, microscopeimaging system 100 determines the number of high-power images to capturebased partly on the digital image storage capacity of the system. Insome embodiments, microscope imaging system 100 determines the number ofhigh-power images to capture based on the knowledge-base and previousexperience capturing images of similar specimen types or diagnostictests.

At 340, high-power images can be captured. Microscope imaging system 100can capture digital images of zelles belonging to clusters containingvaluable content, as determined at 330. This can be done in an ordercorresponding to a ranking of zelles in the clusters, which ranking canbe based on statistical characterization of the zelles in view of thetest being performed (e.g., the percentage of different stain colors,the number of identifiable objects, such as cell nuclei, the ratio ofthe number of cell nuclei of one color versus another, etc.). Moreover,the high-power digital image capture can involve additional automatedanalysis of the captured high resolution images to determine if theymeet specified criteria (e.g., the number of chromosomes in a cellnuclei), and the image capture can then terminate once a sufficientnumber of high-power images have been acquired for the test (e.g., theoperating parameters can specify the number of criteria-meeting samplesto acquire at high power, such as one hundred cells having a specifiednumber of deoxyribonucleic acid (DNA) probe attachments to the nucleusin the case of a fluorescence in situ hybridization (FISH) test).

At 345, acquired images can be stored. The high-power power images canbe saved for future viewing in the memory of microscope imaging system100. Associated with each stored image can be data related to thecoordinates and parameters of the zelles pictured in the digital image.

FIG. 5 shows a flow diagram of method 500 of codifying the knowledgegained from a pathologist's experience of manually identifying regionsof a biological specimen that need to be viewed at higher magnificationand are essential in making a diagnostic evaluation. This codifiedknowledge, or knowledge-base, can be used to set operating parameters at305 of method 300 and can be used to optimize the ability of microscopeimaging system 100 to perform many of the operations of method 300. Inparticular, a knowledge-base can be used to improve the ability ofmicroscope imaging system 100 to identify parameters at 320 of method300 that are best able to form groups or clusters of zelles with similarcharacteristics. A knowledge-base can also be used to improve theability of microscope imaging system 100 to differentiate between theresulting clusters, as at 330 of method 300, and identify those clustersthat are valuable from those that do not contain any useful information.This differentiation can involve differentiating diagnostically valuabletissue from non-diagnostically valuable tissue for the given test.

At 505, a low-magnification image can be displayed. Microscope imagingsystem 100 can display a reconstructed image of microscope slide 200 andsample 230 on display device 114. At 510, an area can be selected forviewing at high-magnification. A pathologist or other expert can viewthe image on display device 114 and select a particular region to viewunder higher magnification. The choice of what region to select is basedon the experience of the pathologist having viewed and diagnosed manysimilar specimens in the past. The pathologist has the ability toquickly recognize an area on a low-magnification image that containsvaluable content necessary to perform a diagnostic evaluation.

At 515, a check is made as to whether there is an image stored of theselected location. The microscope imaging system 100 can determinewhether a high-power image of or including the location selected at 510was previously captured and stored in the memory of microscope imagingsystem 100 at 340 of method 300. If so, the high magnification image canbe displayed at 520. The microscope imaging system 100 can display thehigh-power image selected at 510 on display device 114.

If a high-power image of the selected location was not previouslycaptured and stored, the most similar stored image available can bedetermined at 525. The microscope imaging system 100 can determine whichhigh-power image previously captured and stored in the memory ofmicroscope imaging system 100 at 340 of method 300 is most similar to orexhibits the closest characteristics of the region selected at 510. Thealternative image can be of a zelle from the same cluster as a zellefrom the region originally selected by the pathologist, but may notnecessarily be located near to the region originally selected by thepathologist.

At 530, a determination can be made as to whether the location of thealternative image is acceptable. The microscope imaging system 100 canindicate to the user the location upon microscope slide 200 where thereexists a digital image previously captured and stored in the memory ofmicroscope imaging system 100 that is most similar to or exhibits theclosest characteristics of the region selected at 510. The pathologistor other user can be prompted to indicate whether this alternativeregion will suffice in making their diagnostic evaluation. If so, thealternative high-magnification image can be displayed at 535. Themicroscope imaging system 100 can display the alternative high-powerimage proposed at 530 on display device 114.

If the alterative image is not acceptable, a high-magnification image ofthe originally selected location can be captured at 540. The microscopeimaging system 100 can move microscope slide 200 to the coordinates ofthe region selected at 510 and capture a high-power image of thatlocation and/or display the location live to the user. At 545, the newhigh-magnification image can be displayed. The microscope imaging system100 can display the high-power image captured at 540 on display device114.

At 550, zelle parameters of the selected image can be stored in theknowledge base. The microscope imaging system 100 can store in itsmemory the information and parameters related to those regions asdetermined and selected by the pathologist to be useful in making aspecific diagnostic evaluation on a particular sample 230. Microscopeimaging system can determine which parameters, combination ofparameters, and value ranges form clusters that contain only thosezelles located in the region(s) selected by the pathologist. By havingan expert identify the areas of microscope slide 200 that are criticalto making an effective diagnostic evaluation, and then determining whatthe common characteristics are of those locations, microscope imagingsystem 100 can learn how to automate the process of identifying thoseareas on microscope slide 200 that a pathologist needs to view athigh-power in order to make a diagnosis for similar specimen types andfor similar diagnostic tests. This information can then be used onsimilar diagnostic tests and specimen types in the future to make moreaccurate assessments as to the characteristics of valuable content andwhich parameters form meaningful clusters. The knowledge-base feeds backto the operating parameters for use in future tests as described at 305of method 300.

The processes described above, and all of the functional operationsdescribed in this specification, can be implemented in electroniccircuitry, or in computer hardware, firmware, software, or incombinations of them, such as the structural means disclosed in thisspecification and structural equivalents thereof, including potentiallya program (stored in a machine-readable medium) operable to cause one ormore programmable machines including processor(s) (e.g., a computer) toperform the operations described. It will be appreciated that the orderof operations presented is shown only for the purpose of clarity in thisdescription. No particular order may be required for these operations toachieve desirable results, and various operations can occursimultaneously. For example, the logic flows depicted in FIGS. 3 and 5do not require the particular order shown, sequential order, or that alloperations illustrated be performed, to achieve desirable results. Incertain implementations, multitasking and parallel processing may bepreferable.

The various implementations described above have been presented by wayof example only, and not limitation. Thus, the principles, elements andfeatures described may be employed in varied and numerousimplementations, and various modifications may be made to the describedembodiments without departing from the spirit and scope of theinvention. Accordingly, other embodiments are within the scope of thefollowing claims.

1. An article comprising a machine-readable medium storing instructionsoperable to cause one or more machines to perform operations comprising:processing an image of at least a portion of a scan region including abiological specimen, said processing including differentiating tissue ofgreater interest from tissue of lesser interest in the image based on atest being performed for the biological specimen and based on a clusteranalysis of data from the image; and storing information for the tissueof greater interest, which falls in the scan region.
 2. The article ofclaim 1, wherein the differentiating comprises: defining subimages inthe image based on the test being performed for the biological specimen;determining one or more parameters of the subimages; performing thecluster analysis on the one or more parameters of the subimages; andidentifying one or more areas in the subimages based on results of thecluster analysis, the one or more areas including the tissue of greaterinterest.
 3. The article of claim 2, wherein the defining subimagescomprises specifying subimage dimensions based on the test beingperformed for the biological specimen.
 4. The article of claim 2,wherein the determining comprises determining multiple parameters of thesubimages, and the performing comprises performing a multivariatestatistical cluster analysis on the multiple parameters.
 5. The articleof claim 2, wherein the identifying comprises selecting a proper subsetof resulting clusters based on operating parameters set for the testbeing performed for the biological specimen.
 6. The article of claim 2,wherein the operations further comprise obtaining one or more highermagnification images of the biological specimen in the one or more areasof the subimages, and the information storing comprises saving the oneor more higher magnification images.
 7. The article of claim 2, whereinthe operations further comprise obtaining one or more highermagnification image samples of a cluster until a predefined number ofsamples meeting a specified criteria have been obtained for the cluster.8. The article of claim 2, wherein the operations further compriseobtaining a higher magnification image sample of a cluster, theinformation storing comprises saving the higher magnification imagesample along with a lower magnification image of the cluster andinformation linking the higher magnification image sample with the lowermagnification image, such that the higher magnification image sample isreturned in response to a request for a high power image of a region inthe lower magnification image, wherein the region does not overlap withthe higher magnification image sample but is statistically similar tothe higher magnification image sample according to the cluster.
 9. Thearticle of claim 8, wherein the obtaining the higher magnification imagesample comprises obtaining multiple samples covering representativemembers of the cluster, and the information storing comprises saving thesamples, the lower magnification image and the linking information in asingle file for distribution.
 10. The article of claim 1, the operationsfurther comprising obtaining the image by performing a silhouette scan.11. The article of claim 1, wherein the information storing comprisesretaining high resolution data for the tissue of greater interest, anddiscarding high resolution data for the tissue of lesser interest.
 12. Amethod comprising: obtaining an image of at least a portion of a scanregion including a biological specimen; subdividing the obtained imageinto a plurality of subimages, wherein the subdividing is based on atest being performed for the biological specimen; generating aderivative image wherein image units of the derivative image are derivedfrom respective ones the subimages; performing an automated analysis ofthe derivative image to identify one or more areas of interest for thetest; and storing information for the one or more areas of interest,which fall in the at least a portion of the scan region.
 13. The methodof claim 12, wherein the performing the automated analysis comprises:performing a multivariate statistical cluster analysis; and groupingquadrants of the obtained image based on results of the multivariatestatistical cluster analysis and the test being performed for thebiological specimen.
 14. The method of claim 13, wherein the subdividingthe obtained image comprises specifying subimage dimensions based on thetest being performed for the biological specimen.
 15. The method ofclaim 13, wherein the grouping forms groups of quadrants, and theperforming the automated analysis further comprises selecting a propersubset of the groups based on the test being performed for thebiological specimen to identify the one or more areas of interest. 16.The method of claim 13, wherein the grouping forms groups of quadrants,the performing the automated analysis further comprises determining anumber of sample locations covering representative members of the groupsbased on the test, and the information storing comprises saving lowermagnification image data for the groups and higher magnification imagedata for the sample locations.
 17. The method of claim 13, furthercomprising returning, in response to a request for a high power image ofa first region in the lower magnification image data, at least a portionof the higher magnification image data corresponding to a second regionwith similar characteristics to the first region according to themultivariate statistical cluster analysis.
 18. The method of claim 17,further comprising updating a knowledge-base according to user inputprovided with respect to the at least a portion of the highermagnification image data returned, wherein the updated knowledge-baseaffects future applications of the multivariate statistical clusteranalysis for the test.
 19. An automated imaging system comprising: amicroscope; a controller coupled with the microscope; and a displaydevice coupled with the controller; wherein the controller is configuredto operate the microscope autonomously, to present image data on thedisplay device, and to perform operations including: obtaining an imageof at least a portion of a scan region including a biological specimen;partitioning the obtained image into zelles; determining one or moreparameters of the zelles; performing a cluster analysis on the one ormore parameters of the zelles; differentiating tissue of greaterinterest from tissue of lesser interest in the obtained image based onthe cluster analysis and based on a test being performed for thebiological specimen; and storing more information for the tissue ofgreater interest than information for the tissue of lesser interest. 20.The system of claim 19, wherein the cluster analysis comprises amultivariate statistical cluster analysis, and the zelles comprisetest-dependent zelles.
 21. The system of claim 19, wherein theoperations further include segmenting the zelles into clustersexhibiting similar characteristics among a portion of the one or moreparameters determined to best cluster the zelles according to thecluster analysis.
 22. The system of claim 21, wherein the operationsfurther include determining which clusters contain zelles that mostlikely contain content that is valuable to a pathologist in making adiagnostic evaluation.
 23. The system of claim 21, wherein theoperations further include determining how many high-power images ofzelles to retain for each cluster based on a knowledge-base codifyingprevious test experience.
 24. The system of claim 23, wherein theoperations further include capturing the high-power images of zelles,analyzing the captured high-power images to determine if they meetspecified criteria, and terminating the capturing once a sufficientnumber of high-power images have been acquired for the test according tothe determining how many high-power images of zelles to retain.
 25. Thesystem of claim 23, wherein the operations further include presenting onthe display device, in response to a request for a high power image of afirst region, at least a portion of one or more of the high-powerimages, the at least a portion corresponding to a second region withsimilar characteristics to the first region according to the clusteranalysis.
 26. The system of claim 25, wherein the operations furtherinclude updating the knowledge-base according to user input, wherein theupdated knowledge-base affects future applications of the clusteranalysis for the test.
 27. An apparatus comprising: an interfaceconfigured to connect with a microscope; and a controller configured tosend signals through the interface to operate the microscope and toperform operations including: obtaining an image of at least a portionof a scan region including a biological specimen; partitioning theobtained image into zelles; determining one or more parameters of thezelles; performing a cluster analysis on the one or more parameters ofthe zelles; differentiating tissue of greater interest from tissue oflesser interest in the obtained image based on the cluster analysis andbased on a test being performed for the biological specimen; and storingmore information for the tissue of greater interest than information forthe tissue of lesser interest.
 28. The apparatus of claim 27, whereinthe cluster analysis comprises a multivariate statistical clusteranalysis, and the zelles comprise test-dependent zelles.
 29. Theapparatus of claim 27, wherein the operations further include segmentingthe zelles into clusters exhibiting similar characteristics among aportion of the one or more parameters determined to best cluster thezelles according to the cluster analysis.
 30. The apparatus of claim 29,wherein the operations further include determining which clusterscontain zelles that most likely contain content that is valuable to apathologist in making a diagnostic evaluation.
 31. The apparatus ofclaim 29, wherein the operations further include determining how manyhigh-power images of zelles to retain for each cluster based on aknowledge-base codifying previous test experience.
 32. The apparatus ofclaim 31, wherein the operations further include capturing thehigh-power images of zelles, analyzing the captured high-power images todetermine if they meet specified criteria, and terminating the capturingonce a sufficient number of high-power images have been acquired for thetest according to the determining how many high-power images of zellesto retain.
 33. The apparatus of claim 31, wherein the operations furtherinclude presenting on the display device, in response to a request for ahigh power image of a first region, at least a portion of one or more ofthe high-power images, the at least a portion corresponding to a secondregion with similar characteristics to the first region according to thecluster analysis.
 34. The apparatus of claim 33, wherein the operationsfurther include updating the knowledge-base according to user input,wherein the updated knowledge-base affects future applications of thecluster analysis for the test.